Proceedings  -  23rd  Annual  Conference  -  IEEE/EMBS  Oct.25-28,  2001,  Istanbul,  TURKEY 


1  of  4 


FAST  REGION  DESCRIPTION  METHOD  FOR  CONTENT  BASED  MEDICAL 

IMAGE  RETRIEVAL 

J.  Ruminski,  A.  Nowakowski 

Department  of  Medical  and  Ecological  Electronics,  Technical  University  of  Gdansk,  Gdansk,  Poland 


Abstract  -  A  new  approach  to  segmented  image  representation 
and  description  for  content-based  medical  image  searching  is 
proposed.  Using  the  method  a  table  of  image  runs  is 
dynamically  constructed.  The  table  is  used  by  a  grouping 
process  running  on  the  oversegmennted  image  to  reduce  small 
regions.  In  the  framework  of  the  content-based  querying  the 
table  is  also  applied  to  describe  regions  and  their  contours 
(inner  borders).  The  shape  descriptors  like  an  area,  a  perimeter 
and  a  compactness  are  used  to  indicate  basic  region  properties. 
Application  of  the  method  for  CT  and  MRI  images  produce 
good  results,  which  are  presented  and  discussed  in  this  paper. 
Keywords  -  Region  detection,  RLE,  content  based  retrieval 

I.  Introduction 

With  the  rapid  advances  in  multimodal  medical  imaging  and 
PACS  systems,  it  is  increasingly  important  to  retrieve  images 
from  medical  databases  by  visual  features  either  locally  or 
over  computer  networks.  Content-Based  medical  Image 
Query  (CBIQ)  system  can  be  very  useful  to  assess  the 
quality  of  diagnosis.  It  can  also  be  used  as  an  intelligent 
medical  assistant,  or  can  act  as  a  support  tool,  in  a  pure 
information  browser,  for  guiding  a  search  by  content  in 
electronic  medical  image  archives.  For  the  search  by  content 
functionality,  the  definition  of  appropriate  features  and  their 
syntactic  description,  as  envisaged  in  MPEG7,  is  of  primary 
importance.  Medical  applications  require  to  investigate 
regions  of  an  image  rather  than  the  entire  image.  Region 
detection  and  description  methods  play  a  very  important  role 
for  a  Content-Based  medical  Region  Query  (CBRQ)  system. 
In  this  paper  a  new  approach  to  segmented  image 
representation  and  description  for  content-based  medical 
image  searching  is  proposed. 

II.  RELATED  STUDIES 

Thresholding  is  the  simplest,  fast  yet  often  effective, 
segmentation  method  [1].  In  content  based  searching  the 
most  important  is  to  indicate  the  presence  of  region  presence 
in  the  image,  not  it’s  very  accurate  borders,  which  can  be 
identified  latter  using  more  precise  segmentation  or  region 
detection  algorithms.  Different  thresholding  methods  has 
been  proposed.  The  simplest  method  divides  the  histogram 
domain  into  equal  N  compartments.  Another  methods  are 
looking  for  peaks  and  valleys,  which  are  used  to  set 
thresholds.  Interesting  modification  of  peak  and  valleys 
method  is  the  triangle  algorithm  proposed  in  [2].  Other 
method  is  based  on  modified  image  histogram  created  as  a 
result  of  edge  detection  [1],  Some  optimal  thresholding 
methods  has  been  also  proposed.  These  methods  rely  on 
maximization  (minimization)  of  a  merit  function.  Some 
examples  are  those  using  clustering  operations  [3]  [4]  or 


those  based  on  a  model  [1].  However  most  of  them  uses 
statistical  measures  (e.g.  variance)  to  perform  tests  on 
threshold  selection.  Those  statistical  indicators  are  calculated 
for  each  possible  threshold  location,  so  it  is  a  time  consumed 
procedure.  In  this  paper,  the  shape-based  histogram 
technique  is  proposed  and  used. 

If  a  region  border  is  not  known  but  segments  have  been 
defined  in  the  image,  borders  can  be  uniquely  detected.  A 
very  efficient  method  for  tracking  region  contour  are 
“snakes”  [5].  Active  contours  are  dynamically  modified  by 
minimizing  their  energy.  “Snakes”  are  successfully  applied 
to  singular  regions  within  an  image,  e.g.  a  tumor  area,  but 
not  for  the  complete  set  of  regions  composing  the  image. 
Another,  well  defined  method  for  a  contour  tracking  and 
description  is  the  chain  coding  or  Freeman  coding  [6].  In  this 
method  the  contour  is  followed  from  a  starting  point  by 
encoding  each  pixel  location  in  terms  of  the  relative  direction 
from  its  neighbor.  The  Freeman  chain  codes  is  a  compact 
way  to  represent  a  contour  of  a  region,  since  only  a  starting 
point  coordinates  and  a  set  of  simple  (3  bits)  direction  codes 
are  stored.  The  contour  compression  can  be  even  better  with 
proposed  in  [7]  improvements  of  chain  cods  schemes. 
However  chain  code  has  drawbacks  when  used  for  content 
based  retrieval.  It  must  be  additionally  processed  to  decode 
absolute  region  border,  and  to  calculate  a  region  descriptors. 
Other  methods  for  a  contour  representation  are  often 
constructed  as  graphs  [8].  Graphs  provide  access  to  the  local 
topological  structure  of  the  image,  but  the  shape  of  the  object 
is  only  available  with  additional  examination. 

In  content-based  region  querying  system  proposed  in  [9]  the 
binary  matrix  is  used  to  represent  only  part  of  an  image,  for 
which  feature  data  are  computed.  The  disadvantage  of  this 
method  is  requirement  of  unuseful  background  data  storage 
and  additional  calculations  of  region  pixel  coordinates. 

III.  Methodology 

Before  regions  existing  in  an  image  are  described  they 
must  be  detected.  First  a  fast  thresholding  method  can  be 
used  as  a  segmentation  method.  A  pre-filtering  algorithm  is 
applied  to  an  image  to  eliminate  separate  pixels.  Let’s  define 
the  N8  as  8-connectivity  neighborhood  of  the  processed  pixel 
P(x,y),  minN8  as  a  minimal  value  in  N8,  maxN8  as  a 
maximal  value  in  N8  and  wN9  as  a  vector  of  all  pixel  values 
in  N8  and  processed  pixel  value.  The  pre-filtering  operator 
modifies  the  pixel  P(x,y)  value  according  to  the  schema 

TEST  =  mediana(wN9)-P(x,y); 

If  ( P(x,y)<minN8  ) 

T=K  *  minN8; 
max-false; 
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If  ( P(x,y)>maxN8  ) 

T=maxN8  /  K; 
max=true; 

If  ( |TEST|  >  T ) 

If  (max ) 

P(x,y)=maxN8; 

Else 

P(x,y)=minN8; 

where  K  is  the  scaling  factor  of  the  decision  threshold.  In 
most  cases  of  CT  and  MRI  images  used  in  reported 
experiments  K=1..3. 

Pre-filtered  image  is  suitable  for  thresholding.  The 
modified  histogram  “peak  and  valley”  method  is  used  for 
segmentation.  The  histograms  of  CT  or  MRI  images  have 
usually  two  separate  groups  of  values,  the  foreground  group 
which  corresponds  to  pixels  reconstructed  for  the  measured 
object  (human  body)  and  the  background  group  which 
consists  of  pixels  calculated  for  the  object  outer  space  (air). 
Example  of  MRI  image  histogram  is  presented  in  figure  1 . 
Values  of  an  image  belonging  to  the  foreground  group 
represent  different  tissues  and  are  very  often  spread  around 
of  a  one  value  composing  small  compartments  in  histograms. 
This  feature  is  frequently  used  to  compose  the  image 
presentation  context  (e.g.,  a  window  center  and  width). 
However  in  the  original  CT  and  MRI  image  histograms  local 
fluctuations  are  observed.  In  our  studies  a  morphological 
opening  operator  is  applied  to  a  histogram. 


Fig.  1.  Examples  of  MRI  image  histogram  with  indicated  result  of 
morphological  opening  operator. 

After  elimination  of  small  fluctuations  from  a  histogram 
its  local  minimal  values  are  detected  and  used  for 
thresholding.  As  a  result  a  set  of  threshold  values  are 
produced  whose  total  number  is  usually  less  than  30  (tested 
on  1320  T1  and  T2  weighted  MRI  images  and  1560  CT 
images).  The  number  of  thresholds  can  be  controlled  by 
sequential  elimination  of  these  thresholds  with  the  smallest 
“peaks  to  valley”  gradients.  Each  threshold  produces  a  set  of 
non-adjacent  regions  of  pixels  resulting  in  a  high  number  of 
image  segments.  The  oversegmented  image  requires 
additional  processing  to  merge  those  of  regions  which  are 
most  similar  to  each  other  or  those  which  area  is  smaller  than 
a  given  value.  This  can  be  achieved  by  analyzing  such 
particular  region  properties  like  a  mean  value,  a  location, 


neighborhood,  an  area,  other  properties.  Additionally,  in  the 
framework  of  the  content  based  retrieval,  a  region 
description  method  should  be  fast  and  produce  a  compact 
representation  of  a  region,  suitable  to  store  it  in  a  descriptors 
database. 

The  first  task  in  the  proposed  method  of  region  contour 
detection  and  description  is  generation  of  an  image  run- 
length  codes  (RLC).  With  a  one  pass  scanning  of  an  image  a 
table  consisting  of  N  (N  -  number  of  runs)  runs  is 
constructed.  Each  row  of  the  table  stores  four  run  attributes: 
a  value  of  the  run  V,  the  first  pixel  of  the  run  FP  (From 
Pixel),  the  last  pixel  of  the  run  TP  (To  Pixel),  and  the  region 
label  which  run  belongs  to.  Additionally  indexes  of  first 
occurrences  of  runs  in  a  each  image  raw  are  stored  in  a 
temporary  vector.  Region  labels  are  constructed  dynamically 
according  to  newly  connected  runs.  The  region  labeling 
algorithm  operates  as  follows: 

1 .  label  all  runs  of  the  first  raw, 

2.  store  temporarily  the  first  and  the  last  RLC  table  indexes 
of  all  runs  for  the  first  image  raw, 

3.  move  before  the  first  run  of  the  next  image  raw, 

4.  take  the  next  run, 

5.  test  if  the  current  run  is  adjacent  to  a  any  from  the  previous 
image  raw  with  a  given  value, 

6.  if  “yes”,  add  its  label  to  a  Temporary  Label  Vector  TLV, 

7.  if  all  previous  runs  are  tested  assign  the  smallest  label 
value  to  the  current  run  and  to  all  cells  of  the  external 
vector  S  addressed  by  the  labels  stored  in  the  TLV;  clear 
TLV, 

8.  if  “no”,  assign  a  new  label  to  the  current  run, 

9.  store  temporarily  the  first  and  the  last  RLC  table  indexes 
of  all  runs  for  the  current  image  raw, 

10.  if  there  are  more  unprocessed  runs  go  to  3  else  stop. 

At  the  end  additional  single  pass  re-indexing  of  region 
label  list  must  be  done  basing  external  vector  S.  The 
algorithm  requires  only  the  previous  image  raw  runs  to  test 
for  adjacency  so  it  operates  fast.  Similar  method  of  region 
synthesis  was  introduced  in  [10],  with  application  of 
dynamically  created  region  objects  instead  of  labeling. 

The  result  of  region  synthesis  from  the  RLC  table  gives 
a  set  of  regions  defined  by  runs.  With  such  representation  it 
is  possible  to  efficiently  calculate  the  total  area  of  the  region 
as 

R 

A  =  ^{TPi-FPi  + 1),  (1) 

i= 1 

where  R  -  number  of  runs  belonging  to  the  current  region. 

Having  the  region  area,  the  region  value  and  its  location 
only  its  neighborhood  information  is  additionally  required  to 
start  region  merging  operation.  Region  neighborhood  can  be 
evaluated  using  the  property  of  the  constructed  RLC  table. 
Since  each  run  is  stored  as  a  new  raw,  it  is  very  easy  to  seek 
all  runs  that  are  previous  or  next  to  runs  belonging  to  the 
processed  region.  In  this  way  all  horizontal  neighbors  can  be 
detected  and  ordered  according  to  values  similarity  and  the 
total  number  of  common  border  pixels.  Since  RLC  table 
generation  requires  only  3M  (M  -  number  of  an  image 
pixels)  integer  operations  (comparisons  and  additions  only), 
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it  can  be  repeated  for  columns  of  image  matrix.  Vertical 
runs  are  labeled  into  regions  based  on  comparisons  with 
RLC  table  of  horizontal  runs  (equal  values  of  V  and  FP  for 
horizontal  and  vertical  runs).  Having  the  RLC  table  of 
vertical  runs  theirs  nearest  neighbors  can  be  detected.  In  this 
way  all  horizontal  and  vertical  regions  next  to  the  processed 
region  are  extracted  and  described  by  the  total  number  of 
neighbor  pixels  and  corresponding  values.  Now  it  is  possible 
to  define  regions  merging  algorithm  as 


A.  for  each  region  i  next  to  the  processed  region 

1 .  calculate  gradient  of  corresponding  values 

grad=  V-  reg[i] [0],  (2) 

where  V  is  the  processed  region  value,  reg[i]  [0]  is  the  value 
of  the  i-th  neighbor  region; 

2.  if  the  absolute  value  of  the  gradient  is  lower  than  or  equal 
to  the  threshold  value  Tp;  calculate  new  gradient  value  as 


P  ■  grad 
reg\i][  1]  ’ 


(3) 


where  P  is  the  processed  region  perimeter,  reg[i] [1]  is  the 
common  border  of  the  processed  and  i-th  neighbor  region; 

3.  if  n  V  is  lower  than  current  minimal  value  set  it  as  minimal 
value  and  store  the  neighbor  region  index  ind=i 
B.  if  processed  region  area  is  lower  than  an  arbitrary  set  region 
area  Ap  or  if  minimal  n  V  is  lower  than  an  arbitrary  set  threshold 
value  Vt  join  current  region  and  the  region  identified  by 
reg[ind] [2],  where  reg[ind][2\  is  the  label  of  the  ind-th 
neighbor  region. 


Because  of  its  parameters  the  algorithm  is  universal, 
however  in  our  studies  we  merged  only  those  regions  which 
were  smaller  than  16  pixels  (Tp=0,  Ap=16,  Vt=0).  Required 
perimeter  value  is  the  total  sum  of  all  pixels  belonging  to  the 
inner  border  of  the  region  (external  and  internal  contours). 
Let’s  assign  a  value  “1”  to  all  pixels,  members  of  the 
processed  region  and  a  value  “0”  to  the  rest  of  pixels.  In  our 
studies  perimeter  can  be  defined  as  the  total  number  of  pixels 
with  value  “1”  after  application  of  the  following  formula  to 
the  image  I 

Ip  =  I  -  E(I,N4),  (4) 


where  N4  is  the  4-connectivity  structure  element  and  E(I,N4) 
is  the  morphological  erosion  operator. 

This  definition  is  equivalent  to  the  total  number  (without 
repetitions)  of  FP  and  TP  positions  of  horizontal  and  vertical 
runs  of  the  processed  region.  In  this  way  the  next  region 
descriptor  (perimeter)  can  be  effectively  calculated.  The  last 
region  descriptor  used  for  content  based  region  retrieval 
system  is  the  compactness.  It  is  defined  as 

P2 

C  =  — .  (5) 

A 

The  compactness  is  optimal  for  a  circle.  It  is  possible  to 
calculate  the  compactness  “in  the  fly”  based  on  the  stored 
perimeter  and  area  or  pre-compute  it  and  store  as  additional 
region  parameter. 

The  final  task  of  the  region  description  for  content  based 
region  searching  system  is  definition  of  such  a  region 
representation,  that  it  is  compact  and  enables  fast 


identification  of  the  region  in  the  image  it  belongs  to.  The 
best  method  is  to  represent  and  store  region  borders.  It  can  be 
easily  done  since  the  collections  (without  repetitions)  of  FP 
and  TP  coordinates  of  vertical  and  horizontal  runs  compose 
the  inner  border  of  the  region.  The  simple  rearrangement  of 
this  collection  enable  to  represent  region  borders  using  a 
sequence  of  column  identifiers  of  the  border  pixels  listed 
from  the  first  to  the  last  region  row.  The  region 
representation  algorithm  based  on  RLC  tables  can  be 
described  as 

1 .  for  each  row  store  all  run  elements  which  belong  to  the 
border; 

2.  if  within  a  one  raw  two  or  more  separate  mns  of  the 
region  exist  store  the  value  “-1”  between  the  last  and  the 
first  coordinates  of  corresponding  border  elements  of 
runs, 

3.  if  the  first  column  coordinate  of  the  next  row  run  is 
grater  than  the  last  column  coordinate  of  the  current  row 
run  store  the  “-2”  value  before  the  next  row  run. 

This  results  with  the  set  of  border  pixels  coordinates  related 
to  columns.  Additionally  the  identifier  of  the  first  region  row 
has  to  be  stored.  Together  with  the  value  of  the  region  the 
full  description  of  the  region  is  created.  The  simple  example 
of  the  region  description  is  presented  in  figure  2.  Description 
can  be 


horizontal  RLC  table 


V 

FP 

TP 

S 

1 

0 

2 

0 

1 

0 

2 

0 

1 

0 

1 

0 

0 

2 

2 

1 

0 

0 

1 

1 

1 

2 

2 

0 

R={value,  row,  border  pixels} 
R0={1 ,0,0,1 ,2, 0,2, 0,1  ,-2,2} 
R1={0,2,2,0,1} 


vertical  RLC  table 
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FP 

TP 

S 

1 

0 

2 

0 

0 

3 

3 

1 

1 

0 

2 

0 

0 

3 

3 

1 

1 

0 

1 

0 

0 

2 

2 

1 

1 

3 

3 

0 

Fig.  2.  Example  of  region  description  result 


stored  as  rows  in  tables  (e.g.,  in  a  relational  database)  or  as 
persistent  objects  (in  an  object  oriented  database).  It  is  also 
possible  to  add  additional  fields  to  a  region  row  or  an  object 
to  store  descriptor  values  like  area,  perimeter,  compactness, 
etc.  In  this  case  the  example  of  region  description  class  in 
object  oriented  design  is 

class  Regionj 
int  value; 
int  area, 
int  perimeter; 
int  compactness; 
int  first_row; 
int  border[]; 

} 

However  the  interesting  feature  of  the  proposed  region 
description  is  that  descriptor  values  can  be  calculated  “in  the 
fly”  based  on  the  introduced  region  representation.  Simple, 
perimeter  is  the  total  number  of  all  positive  values 
representing  the  region  border.  The  area  is  equal  to  the 
perimeter  extended  with  all  pixels  between  runs,  stored  in  the 
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border  description  (in  the  presented  example  of  R0,  the  0,2 
gives  additional  pixel  of  1  to  the  area). 

III.  Results  and  conclusion 

The  presented  method  of  region  description  for  content 
based  searching  system  was  verified  using  simple  test 
images  and  real  medical  images  (1320  T1  and  T2  weighted 
MRI  images  and  1560  CT  images).  Medical  images  were 
stored  in  primary  captured  DICOM  format,  so  the  original 
representation  of  data  was  used  (usually  12  bits).  Particular 
algorithms  were  implemented  in  Java  programming  language 
using  Sun  JDK  1.2.  Running  the  application  on  different 
platforms  very  fast  results  of  the  described  method  were 
produced.  Since  specification  of  time  depends  on  platform 
configuration,  it  is  more  useful  to  calculate  the  number  of 
operations.  In  the  proposed  method  the  total  cost  of 
operations  can  be  calculated  as 

1.  RLC  tables  generation  costs:  3M+M=4M, 

2.  region  extraction  from  RLC  table:  5N, 

3.  region  description  generation:  5N, 

which  gives  the  total  number  of  4M  (M  -  number  of  image 
pixels)  +  10N  (N-number  of  runs)  operations.  Since  most  of 
them  are  scanning  operations  (2M+4N),  and  all  of  them  are 
simple  comparisons  and  additions  of  integer  numbers,  the 
method  works  very  efficiently.  In  the  presented  cost 
estimation  additional  assumption  was  made  that  the  number 
of  runs  in  the  horizontal  RLC  table  is  equal  to  the  number  of 
runs  in  the  vertical  table. 

In  figure  3  the  test  image  presented  in  [10]  and  the  result  of 
its  processing  are  presented.  According  to  8 -connectivity 
test  used  in  our  method  there  are  4  different  regions  in  the 
test  image.  The  “foreground”  region  indicated  with  gray  has 
external  and  internal  borders.  However  for  content  based 
searching  system  it  is  useful  to  work  with  combined  borders 
(a  perimeter  defined  above),  witch  represent  the  region.  With 
this  assumption  presented  method  does  not  requires  any  time 
consumed  “walking”  operation  so  it  works  faster. 


Fig.  3.  Example  of  an  inner  border  extraction  result 

In  figure  4  the  image  from  the  created  medical  image 
database  is  presented  together  with  the  effect  of  thresholding 
and  with  the  region  generated  from  its  description. 

The  proposed  method  can  be  effectively  used  for  content 
based  region  retrieval  system  in  different  ways.  First,  it  is 


Fig.  4.  Example  of  region  description  result.  From  left:  CT  image  scaled  to 
its  full  range,  image  after  thresholding,  a  one  region  generated  from  its 
description. 


possible  to  generate  shape  descriptors  used  in  comparison 
tests;  second,  region  location  can  be  encoded  used  to  a 
region  identification  in  the  retrieved  image.  Additional 
advantage  of  the  method  is  the  possibility  of  a  region 
extraction  from  the  segmented  image  or  from  the  image 
generated  as  a  result  of  pattern  matching.  It  can  be  used  then 
to  generate  a  database  of  regions  for  particular  image 
databases  or  for  a  collection  of  templates  used  in  pattern 
matching. 
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